Focus detection apparatus, image pickup apparatus, image pickup system, focus detection method, and non-transitory computer-readable storage medium

ABSTRACT

A focus detection apparatus performs focus detection by a phase difference method using an image pickup element including first and second pixels, the focus detection apparatus includes a correlation data calculator which calculates correlation data between pixel data obtained from the first pixels and the second pixels in ranges of image data, a detector which detects a saturated pixel having a level of at least a predetermined value in each of the ranges, an adding processor which performs an addition processing of the correlation data calculated in each of the ranges based on a detection result, and a defocus amount calculator which calculates a defocus amount based on a result of the addition processing, and the adding processor performs the addition processing using correlation data obtained from a first range in which the number of the saturated pixels is less than a predetermined number.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a focus detection apparatus whichperforms focus detection by a phase difference method using an imagepickup element including a plurality of photoelectric conversionportions sharing one microlens.

2. Description of the Related Art

A focus detection method is well known in the related art, in whichtaking an image and focus detection by a phase difference method areperformed using a solid-state image pickup element including a pluralityof photoelectric conversion portions sharing one microlens. JapanesePatent Laid-open No. 2001-83407 discloses a configuration in which twophotoelectric conversion portions divided with respect to one microlensobtain pupil-divided image signals to perform a focus detectionprocessing by the phase difference method. In addition, Japanese PatentLaid-open No. 2001-83407 discloses a configuration in which a valueobtained by adding the outputs of two photoelectric conversion portionscorresponding to the same microlens is handled as an output of a pixelto obtain an image signal as an image pickup signal.

Japanese Patent Laid-open No. 2007-133087 discloses a configuration inwhich image signals from a plurality of photoelectric conversionportions sharing one microlens are selectively read out as an A-image ora B-image to perform a correlation calculation. Japanese Patent No.4691930 discloses a configuration in which some of a plurality ofdivided photoelectric conversion portions are read out through anon-destructive readout and then a synthesis component value of eachphotoelectric conversion portion is read out. Then, the number ofreadout pixels is reduced by estimating other divided photoelectricconversion portion based on a difference between the synthesis componentvalue and a pixel value of a part of the photoelectric conversionportions.

Since an output value of the image signal as the image pickup signal isgenerated by using an addition value of the plurality of dividedphotoelectric conversion portions in the configuration as disclosed inJapanese Patent Laid-open No 2001-83407 and Japanese Patent Laid-openNo. 2007-133087, an upper limit value of a light amount allowed to bephoto-electrically converted in each photoelectric conversion portion issmaller compared with that of the configuration in which thephotoelectric conversion portion is not divided. That is, when lighthaving intensity which exceeds a saturation level is exposed on only oneof the photoelectric conversion portions, a charge for the saturationare not reflected in the image signal as the image pickup signal, andtherefore the image signal as the image pickup signal is not correctlygenerated.

In addition, it is considered to calculate a correlation waveform foreach of a plurality of ranging regions in which the change ofcorrelation value in a column direction is obtained, from a signal ofeach pupil-divided pixel of each pixel when performing the focusdetection by the phase difference method using the image pickup element.In this case, the plurality of correlation waveforms calculated from theplural ranging regions are added to be able to calculate a shift amountof a focus by obtaining an image shift amount based on the addedcorrelation waveform.

However, when the correlation waveform is calculated in a regionincluding saturated pixels at the time of performing the focus detectionby the phase difference method using the image pickup element, the imageshift amount obtained based on the correlation waveform becomes zero andan erroneous focus shift amount is calculated, and thus, focus detectionaccuracy is deteriorated.

BRIEF SUMMARY OF THE INVENTION

The present invention provides a focus detection apparatus, an imagepickup apparatus, an image pickup system, and a focus detection methodcapable of reducing a deterioration of focus detection accuracy evenwhen a saturated pixel is included. The present invention also providesa non-transitory computer-readable storage medium storing a programwhich causes a computer to execute the focus detection method.

A focus detection apparatus as one aspect of the present inventionperforms focus detection by a phase difference method using an imagepickup element including a plurality of first pixels and a plurality ofsecond pixels, each pair of the first and second pixels sharing onemicrolens, the focus detection apparatus includes a correlation datacalculator configured to calculate correlation data between pixel dataobtained from the plurality of first pixels and pixel data obtained fromthe plurality of second pixels in a plurality of ranges of image dataobtained from the image pickup element, a detector configured to detecta saturated pixel having a level of at least a predetermined value ineach of the plurality of ranges, an adding processor configured toperform an addition processing of the correlation data calculated ineach of the plurality of ranges, based on a detection result of thesaturated pixel by the detector, and a defocus amount calculatorconfigured to calculate a defocus amount based on a result of theaddition processing, and the adding processor performs the additionprocessing using correlation data obtained from a first range in whichthe number of the saturated pixels is less than a predetermined number.

An image pickup apparatus as another aspect of the present inventionincludes the focus detection apparatus.

An image pickup system as another aspect of the present inventionincludes the image pickup apparatus and a lens apparatus removablymounted on the image pickup apparatus, and the image pickup apparatusperforms a drive control of the lens apparatus based on a signalobtained from the focus detection apparatus.

A focus detection method as another aspect of the present inventionperforms focus detection by a phase difference method using an imagepickup element including a plurality of first pixels and a plurality ofsecond pixels, each pair of the first and second pixels sharing onemicrolens, the focus detection method includes the steps of calculatingcorrelation data between pixel data obtained from the plurality of firstpixels and pixel data obtained from the plurality of second pixels in aplurality of ranges of image data obtained from the image pickupelement, detecting a saturated pixel having a level of at least apredetermined value in each of the plurality of ranges, performing anaddition processing of the correlation data calculated in each of theplurality of ranges, based on a detection result of the saturated pixel,and calculating a defocus amount based on a result of the additionprocessing, and the addition processing is performed using correlationdata obtained from a range in which the number of the saturated pixelsis less than a predetermined number.

A non-transitory computer-readable storage medium as another aspect ofthe present invention stores a program which causes a computer toexecute each step of the focus detection method.

Further features and aspects of the present invention will becomeapparent from the following description of exemplary embodiments withreference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic configuration diagram of an image pickup elementin each embodiment.

FIG. 2 is a schematic diagram of a pixel of a pixel array in eachembodiment.

FIG. 3 is a schematic diagram of the pixel array in each embodiment.

FIG. 4 is a conceptual diagram in which light beam emitted from an exitpupil of an image pickup lens enters an image pickup element in eachembodiment.

FIG. 5 is a diagram of focus detection by a focus detection apparatus ineach embodiment.

FIGS. 6A and 6B are schematic diagrams of ranging regions of the imagepickup element in each embodiment.

FIG. 7 is a flowchart illustrating a ranging calculation processing ineach embodiment.

FIG. 8 is a flowchart illustrating a calculation processing of acorrelation waveform in each embodiment.

FIGS. 9A to 9C are diagrams illustrating relations between correlationwaveforms and focus states in each embodiment.

FIGS. 10A to 10D are diagrams illustrating the correlation waveformsdepending on the presence of a saturated pixel in each embodiment.

FIG. 11 is a flowchart illustrating an addition processing of thecorrelation waveform in Embodiment 1.

FIG. 12 is a flowchart illustrating an addition processing of decorrelation waveform in Embodiment 2.

FIG. 13 is a schematic diagram of an image pickup system in eachembodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, embodiments of the present invention will be described withreference to the drawings. Note that the same components will be denotedby the same reference numerals to avoid a duplicated description in eachdrawing.

Embodiment 1

First of all, referring to FIG. 1, a configuration of an image pickupelement in Embodiment 1 of the present invention will be described. FIG.1 is a schematic configuration diagram of the image pickup element inthe present embodiment. In FIG. 1, an image pickup element 100 includesa pixel array 101 (a pixel portion), a vertical selecting circuit 102which selects a row in the pixel array 101, and a horizontal selectingcircuit 104 which selects a column in the pixel array 101. In addition,the image pickup element 100 includes a readout circuit 103 which readsout a signal of a pixel selected by the vertical selecting circuit 102among pixels of the pixel array 101, and a serial interface 105 whichexternally determines an operation mode of each circuit and the like.

The readout circuit 103 includes components such as a memory for storingthe signal, a gain amplifier, and an AD converter for each column. Theimage pickup element 100 includes a timing generator or a controlcircuit which provides a timing signal to, for example, the verticalselecting circuit. 102, the horizontal selecting circuit 104, or thesignal readout portion 103 in addition to the components illustrated inFIG. 1. Typically, the vertical selecting circuit 102 selectssequentially a plurality of rows of the pixel array 101, and the readoutcircuit 103 reads out the selected pixel signal. The horizontalselecting circuit 104 sequentially selects the plurality of pixelsignals read out by the readout circuit 103 for each column.

Subsequently, referring to FIG. 2, a configuration of a pixel in thepixel array 101 will be described. FIG. 2 is a schematic diagram of thepixel in the pixel array 101. Reference numeral 201 denotes one pixel inthe pixel array 101. Reference numeral 202 denotes a microlens. Thepixel 201 includes the microlens 202. Reference numerals 203 and 204denote photo diodes. The pixel 201 includes two photo diodes(hereinafter, referred to as a “PD”) 203 and 204. Furthermore, the pixel201 is configured to include, for example, a pixel amplifying amplifierused to have the signals of the PDs 203 and 204 be read out by thecolumn readout circuit 103, a selection switch which selects the row,and a reset switch which resets the signals of the PDs 203 and 204, inaddition to the components illustrated in FIG. 2.

Subsequently, referring to FIG. 3, an entire configuration of the pixelarray 101 will be described. FIG. 3 is a schematic diagram of the pixelarray 101. The pixel array 101 is configured such that a plurality ofpixels illustrated in FIG. 2 are arranged in a two-dimensional array toprovide a two-dimensional image. Reference numerals 301, 302, 203, and304 respectively denote pixels. Reference numerals 301L, 302L, 303L, and304L respectively denote photo diodes (PDs), which correspond to the PD203 in FIG. 2. Reference numerals 301R, 302R, 303P, and 304Rrespectively denote photo diodes (PDs) which correspond to the PD 204 inFIG. 2.

Next, referring to FIG. 4, a light receiving state in the image pickupelement 100 having the pixel configuration illustrated in FIG. 3 will bedescribed. FIG. 4 is a conceptual diagram in which light beam emittedfrom an exit pupil of an image pickup lens enters the image pickupelement 100. In FIG. 4, a cross-section of the pixel array 101 isillustrated. Reference numeral 402 denotes a microlens, and referencenumeral 403 denotes a color filter. Reference numeral 406 denotes anexit pupil of the image pickup lens. In the present embodiment, anoptical axis 409 is a center of the light beam emitted from the exitpupil 406 with respect to two photo diodes (PDs 203 and 204) sharing onemicrolens 402. Light emitted from the exit pupil 406 enters the imagepickup element 100 around the center of the optical axis 409.

Reference numerals 407 and 408 respectively denote regions (partialregions) of the exit pupil of the image pickup lens different from eachother. Reference numeral 410 denotes the outmost ray of light passingthrough the region 407 of the exit pupil 406. Reference numeral 411denotes the outmost ray of light passing through the region 408 of theexit pupil 406. As illustrated in FIG. 4, with respect to the light fluxemitted from the exit pupil 406, the light beam upper than the opticalaxis 409 (the light passing through the region 407) enters the Pp 204,and the light beam lower than the optical axis 409 (the light passingthrough the region 408) enters the PD 203. In other words, the PD 203and PD 204 respectively receive the light passing through the regions(regions 407 and 408) of the exit. pupil 406 of the image pickup lensdifferent from each other.

In the present embodiment, as illustrated in FIGS. 2 and 4, theconfiguration where the microlens 202 (402) shares two photo diodes (PDs203 and 204) is adopted. However, the present embodiment is not limitedto this, and focus detection can be performed by a phase differencemethod even in a configuration where a certain pixel is only providedwith a one-side PD and an adjacent pixel is only provided with anopposite-side PD. In addition, the focus detection can be performed bythe phase difference method even in a case where a light blocking layeror the like is provided to block the light entering from one side of themicrolens 202. Furthermore, if the image pickup element has aconfiguration capable of separately (independently) obtaininginformation on the light beam emitted from the exit pupil 406 of theimage pickup lens, and includes two-dimensionally arrayed pixels, thepresent embodiment is not limited to the configuration described above.

An image pickup apparatus of the present embodiment includes the focusdetection apparatus, and obtains images of the regions different fromeach other of the exit pupil in the image pickup lens by the imagepickup element 100 to perform she focus detection by the phasedifference method. In other words, the image pickup apparatus (the focusdetection apparatus) performs the focus detection by the phasedifference method using the image pickup element 100 including a firstpixel (PD 202) and a second pixel (PD 203) sharing one microlens 202.

Next, referring to FIG. 5 and FIGS. 6A and 6B, a processing of drivingthe image pickup lens based on a shift amount of two images will bedescribed. First of all, referring to FIG. 5, the focus detection (theranging operation) by the focus detection apparatus in the presentembodiment will be briefly described. FIG. 5 is a diagram of the focusdetection by the focus detection apparatus. In FIG. 5, an upper partillustrates a pixel arrangement in which pixels are arranged in one rowof a focus detection region (a ranging region) of the image pickupelement 100, and a lower part illustrates each image at each focusposition in a state where a horizontal axis is a pixel position and avertical axis is a signal level. addition, (a) of FIG. 5 illustrates anin-focus state, (b) of FIG. 5 illustrates a front focus state, and (c)of FIG. 5 illustrates a rear focus state. The image pickup element 100is configured such that A-line pixels and B-line pixels receiving thelight emitted from the different exit pupils of the image pickup lensare disposed an two-dimensional array.

Referring to FIG. 3, an A-line (a first pixel array) is configured byPDs 301L, 302L, 303L, and 304L (A-image pixel, or a first pixel) out ofa row 305. A B-line (a second pixel array) is configured by PDs 301R,302R, 303R, and 304R (B-image pixel, or a second pixel). As illustratedin FIG. 5, the outputs of the A-line and B-line change such thatintervals (image intervals) of two images are different, in accordancewith any one of the in-focus state, the front focus state, and the rearfocus state. A focus is adjusted by moving a focus lens of the imagepickup lens so that the image interval at the in-focus state isobtained. That is, a moving distance of the focus lens can be obtainedby calculating the shift amount of two images.

Subsequently, referring to FIGS. 6A and 6B, the focus detection region(the ranging region) on the image pickup element 100 will be described.FIGS. 6A and 6B are schematic diagrams of the ranging region on theimage pickup element 100. FIG. 6A illustrates the ranging region. In thepresent embodiment, the ranging region (a range of the rangingcalculation) is set so as to have an X-direction (a horizontaldirection) and a Y-direction (a vertical direction) around a center 601of the ranging region, and the X-direction is within a range from a p-thcolumn to a q-th column and the Y-direct ion is within a range from ar-th row to a s-th row. In addition, a shift amount is within a rangefrom −imax to +imax. In the present embodiment, the ranging region wherethe ranging is possible is a region 603 which is configured to alsoinclude the shift amount. In the region 603, the focus detectionapparatus compares the output of the A-line with she output of theB-line.

FIG. 6B illustrates a state where the ranging region in the pixel array101 is set at a position different from that of the ranging regionillustrated in FIG. 6A. As illustrated in FIG. 6B, the rangingcalculation (the focus detection calculation) can be performed at anarbitrary position on a screen by displacing (by moving) the rangingregion.

Next, referring to FIG. 7 to 11, a ranging calculation processing (afocus detection method) in the present. embodiment will be described.FIG. 7 is a flowchart illustrating the ranging calculation processing,and illustrates a series of processing from the ranging calculation tothe driving of the lens (the image pickup lens). In the presentembodiment, a processing within a range of the ranging calculation asillustrated in FIG. 6A is described. Each step in FIG. 7 is performedbased on an instruction from a controller (for example, a controller1309) of the image pickup apparatus (the focus detection apparatus).

In FIG. 7, when the ranging calculation starts, first of all, in stepS701, the process proceeds to a subroutine which calculates acorrelation waveform Cm(I) of the ranging region that is a range of theranging calculation. Referring to FIG. 8, the subroutine of step S701,that is, a calculation method of the correlation waveform Cm(I) will bedescribed. FIG. 8 is a flowchart illustrating the calculation method ofthe correlation waveform Cm(I) in the ranging region (the focusdetection region) illustrated in FIG. 6A, based on image data (an imagesignal) obtained from the image pickup element 100. Each step in FIG. 8is performed based on an instruction of the controller (for example, thecontroller 1309) of the image pickup apparatus (the focus detectionapparatus).

In FIG. 8, when the calculation of the correlation waveform starts,first of all, an initial row is selected as Y=r in step S800. Next,saturated pixel detection is performed in step S801. The saturated pixelis determined based on whether each pixel (a signal value of each pixel)reaches a certain saturated level which is previously set. At this time,the controller 1309 (a detector) detects the saturated pixel having alevel of at least a predetermined value in each of a plurality ofranges. In step S801, when the pixel reaches the saturated level, thepixel is determined as a saturated pixel and a saturation hit is givento the pixel signal. Furthermore, instead of giving the saturation bit,a part of pixel signals may be treated as a saturation bit, and thepolarity thereof may not be restricted.

Next, in step S302, the relation of Iy=−Imax is set. Here, since the rowis Y=r, an image shift amount of the r row is obtained. Subsequently, instep S803, image data of the B-image (B-line) are shifted by Iy pixels.Then, in step S304, the correlation value (the correlation waveform) atshe time of the Iy pixel shift of the A-image (image data of A-line) andthe B-image (image data of B-line) is obtained. Specifically, acorrelation value (a correlation waveform Cm(Iy)) is calculated byobtaining an absolute value of a difference between two images (adifference of the image data) in each pixel of the A-line and theB-line, as represented by the following Expression (1).

$\begin{matrix}{{{Cm}({Iy})} = {\sum\limits_{x = p}^{q}{{A_{x} - B_{x + {Iy}}}}}} & (1)\end{matrix}$

In Expression (1), A, and B, indicate outputs of x-coordinates of theA-line and the B-line, respectively, in a designated row. That is,Cm(Iy) is a sum of the absolute values of the differences between theimage data of the A-line and the image data of the B-line when theB-line in the ranging region “m” is shifted by the Iy pixels.

In addition, the correlation value in the present embodiment is notlimited to she calculation using Expression (1), but may be calculatedusing, for example, the following Expression (2).

$\begin{matrix}{{{Cm}({Iy})} = {\sum\limits_{x = p}^{q}{{A_{x + {Iy}} - B_{x - {Iy}}}}}} & (2)\end{matrix}$

In Expression (2), by not only shifting the image data of the B-line butalso concurrently shifting the image data of the A-line in an oppositedirection, the sum of the absolute values of the differences betweenthese image data is obtained.

In this case, in step S803, the image data of the A-line is shifted bythe Iy pixels, and the image data of the B-line is shifted by −Iypixels.

In addition, the correlation value in the present embodiment can becalculated using a pixel value greater of the pixel values (image data)of each pixel as represented by the following Expression (3), other thanthe calculation based on the absolute value of the difference betweenthe image data of the A-line and the image data of the B-line.

$\begin{matrix}{{{Cm}({Iy})} = {\sum\limits_{x = p}^{q}{\max \left( {A_{x},B_{x + {Iy}}} \right)}}} & (3)\end{matrix}$

In Expression (3), max (A, B) represents that the larger value of A andB is selected. Although no specific expression is described herein, itis possible to obtain the correlation value (the correlation waveform)using the calculation in which the smaller value of A and B is selected.Thus, in the present embodiment, de correlation waveform (thecorrelation data) in each of the plurality of ranges is calculated byrelatively displacing (shifting) a first pixel array including aplurality of first pixels and a second pixel array including a pluralityof second pixels in a plurality of ranges of the focus detection region(the image data obtained from the image pickup element 100). In otherwords, correlation data calculator calculates correlation data betweenpixel data obtained from the plurality of first pixels and pixel dataobtained from the plurality of second pixels.

The calculation of the correlation waveform is performed by, forexample, the controller 1309 (a correlation data calculator) in FIG. 13.In the embodiment, the calculation method of the correlation value instep S804 is not particularly limited.

Next, in step S805, one pixel is displaced by substituting Iy+1 for Iy.Subsequently, in step S806, it is determined whether Iy is greater thanImax (Iy>Imax). When Iy is greater than Imax, the flow proceeds to stepS807. On the other hand, when Iy is not more than Imax, the flow returnsto step S803 and steps S803 to S805 are repeated.

Next, in step S807, Cm (Iy)+Cm(I) is substituted for the Cm(I). In otherwords, the controller 1309 (an adding processor) performs the additionprocessing of the correlation waveform calculated in each of theplurality of ranges of the focus detection region. That is, the addingprocessor performs the addition processing of the correlation datacalculated in each of the plurality of ranges based on the detectionresult of the saturated pixel by the detector.

Subsequently, in step S808, (Y+1) is substituted for Y to move on onerow. Then, in step S809, it is determined whether the relation of Y>s issatisfied. In a case where the relation of Y>s is satisfied, thecalculation of the correlation waveform is ended. On the other hand, ina case where the relation of Y>s is not satisfied, the flow returns tostep S802 and steps S802 to S808 are repeated. In step S807 of thepresent embodiment, the correlation waveform Cm(I) is generated byadding the correlation waveform Cm(Iy) of each row. Then, this step isrepeatedly performed from the r-th row to the s-th row, the correlationwaveform Cm(I) obtained by adding values in all rows is calculated basedon the correlation waveform Cm(Iy) of each row.

Subsequently, referring to FIGS. 9A to 9C, the correlation waveform willbe described. FIGS. 9A to 9C are diagrams of the correlation waveform.FIG. 9A indicates the in-focus state, FIG. 9B indicates the front focusstate, and FIG. 9C indicates the rear focus state. The correlationwaveform Cm(I) is the correlation value between the image data of theA-line and the image data of the B-line obtained by shifting the linesby a shift amount I. When the sum of the absolute values of thedifferences between these image data is used to obtain the correlationvalue, the shift amount I at a location where the output of thecorrelation waveform Cm(I) is lowest is a shift amount I where thecorrelation is maximized.

As illustrated in FIG. 9A, the shift amount I where the correlation ismaximized in the correlation waveform Cm(I) at the time of the in-focusstate is a shift amount (I=0) at a location where the output of thecorrelation waveform Cm(I) is the lowest. On the other hand, asillustrated in FIGS. 9B and 9C, when the focus state is an out-of-focusstate, an image shift amount depending on the shift amount of the focusis the shift amount I. In other words, the shift amount I is equivalentto the image shift amount, and the relation of “shift amount I=imageshift amount I” is substantially satisfied.

Next, referring to FIGS. 10A. to 10D, step S807 in FIG. 8 will bedescribed in detail. FIG. 10A illustrates a relation between a pixelvalue (an intensity distribution) of the A-image and the B-image at theline including the saturated pixels in the range 602 of the rangingcalculation and a

horizontal position. The A-image is indicated by a solid line, and theB-image is indicated by a dotted line. The pixel values of both theA-image and B-image are larger at the line with the saturated pixels.

FIG. 10B illustrates a relation between. SAD (sum of absolutedifferences: plots on a logarithmic scale) at the line including thesaturated pixels in the range 602 of the ranging calculation and adefocus amount. The defocus amount is calculated based on the result ofthe addition processing by the controller 1309 (a defocus amountcalculator). The correlation value increases in a zone with thesaturated pixel, and the value of SAD becomes smaller at a positionwhere the defocus amount is ±0. However, a total value of SAD is largerdue to the influence of the saturated pixel as compared to a line (FIG.10D) which has no saturated pixel, regardless of the defocus amount.

FIG. 10C illustrates a relation between a pixel value (an intensitydistribution) of the A-image and the B-image at the line not includingshe saturated pixel in the range 602 of the ranging calculation and ahorizontal position. FIG. 10D illustrates a relation between the SAD atthe line not including the saturated pixel in the range 602 of theranging calculation and the defocus amount. As illustrated in FIG. 10D,the value of SAD is small when the defocus amount is +10. However, sincethe value of SAD is not influenced by the saturated pixel, the value ofSAD is smaller as compared with the line (FIG. 10B) including thesaturated pixel at a position where the defocus amount is +10.Therefore, since the defocus amount becomes zero (0) by adding thecorrelation waveform of the line including the saturated pixel, thedefocus amount to be obtained is a value different from the value of+10. Even in one line, when the addition on processing of thecorrelation waveform is performed at the line including the saturatedpixel, the value of the correlation waveform totally increases, and thusthere is a possibility of an erroneous detection of defocusinginformation.

Next, referring to FIG. 11, the addition processing of the correlationwaveform in the present embodiment will be described. FIG. 11 is aflowchart illustrating the addition processing of she correlationwaveform. Each step in FIG. 11 is performed based on an instruction ofthe controller (for example, the controller 1309) of the image pickupapparatus (the focus detection apparatus).

First of all, in step S1101, the controller 1309 (a determination unit)determines whether the saturated pixel are included in the A-line (thefirst pixel array) or the B-line (the second pixel array) in each of theplurality of ranges of the focus detection region (determination of asaturated line). That is, it is determined whether the saturated pixelis detected in step S801 of FIG. 8. When it is determined that the lineis the saturated line, i.e. the saturated. pixel is detected, in stepS1101, the flow proceeds to step S1102.

In step S1102, the controller 1309 (an adding processor) performs theaddition processing using the correlation data (the correlationwaveform) obtained from the range in which The number of saturatedpixels is less than a predetermined number. In the present embodiment,when the saturated pixel is included in the first pixel array or thesecond pixel array in a first range (an I-th row) out of the pluralityof ranges, the adding processor performs the addition processing so asto reduce the influence of the correlation waveform calculated in thefirst range. Specifically, in the present embodiment, an interpolationprocessing of the saturated line is performed for the correlationwaveform including the saturated pixel, using the previous or nextcorrelation waveform. With respect to the interpolation processing ofthe saturated line, in a case of interpolating the correlation waveformfor the previous one line, that is, in a case of performing apre-interpolation, the interpolation is performed as represented by thefollowing Expression (4).

Cm(I)=Cm(I−1)  (4)

Furthermore, in a case of interpolating the saturated line using thecorrelation waveform for the previous and next one lines, theinterpolation is performed as represented by the following Expression(5).

Cm(I)=(Cm(I−1)+Cm(I+1))/2  (5)

Thus, in the present embodiment, when the saturated pixel is included inthe first pixel array or the second pixel array in the first range, theadding processor uses the correlation waveform of the first pixel arrayor the second pixel, array of the second range adjacent to the firstrange as a correlation waveform (correlation data) of the first range.In other words, the adding processor uses the correlation data of thefirst pixel or the second pixel in the second range adjacent to thefirst range as a correlation data in the first range, for the range inwhich the number of saturated pixels among the first pixels or thesecond pixels is at least a predetermined number. For example, when thefirst range is the I-th row, the second range is the (I+1)-th row or the(I−1)-th row.

In the present embodiment, the interpolation processing of thecorrelation waveform may also be performed within a range where thesaturated pixel exists. In this case, when the saturated pixel isincluded in the first pixel array or the second pixel array in the firstrange, the adding processor uses the correlation waveform in the secondrange adjacent to the first range as a correlation waveform in the firstrange, for the range where the saturated pixel exists in the firstrange. In other words, when the saturated pixel is included in the firstpixels or the second pixels, the adding processor uses the correlationwaveform in the second range adjacent to the first range as acorrelation waveform in the first range, for the range where thesaturated pixel exists in the first range. For example, it is possibleto interpolate the correlation waveform of the zone with the saturatedpixel in FIG. 10A using the correlation waveform of the zone without thesaturated pixel in FIG. 10C.

Next, in step S1103, a weighting addition processing of the correlationwaveform is performed as represented by the following Expression (6).

Cm(I)=Cm(I)+k×Cm(Iy)  (6)

In Expression (6), k denotes a coefficient of a weighting addition andtakes a value between 0.0 and 1.0.

In this case, when the saturated pixel is included in the first pixelarray or the second pixel array in the first range, the adding processorperforms the addition processing by multiplying the coefficient k (aweight) depending on the number of saturated pixels by the correlationwaveform in the first range. It is preferred that the coefficient kcomes closer to the value of one as the number of the saturated pixelsis smaller, and that the coefficient k comes closer to the value of zeroas the number of the saturated pixels is larger. When all of the pixelsin a certain line are the saturated pixels as a result of the detectionof the saturated pixels in step S801 of FIG. 8, the coefficient k mayalso be zero.

In the present embodiment, the first pixels (A-image pixels) and thesecond pixels (B-image pixels) receive the light beam passing throughthe regions different from each other in a pupil dividing direction ofthe exit pupil to generate the signal to be used for the calculation ofthe correlation waveform. Then, the adding processor adds each of thecorrelation waveforms of the plurality of ranges in a directionorthogonal to the pupil dividing direction. In the present embodiment,the pupil dividing direction of the A-image and the B-image is definedas a row direction and the adding direction. of the correlation waveformis defined as a column direction, but the embodiment is not limited tothis. Conversely, the pupil dividing direction of the A-image and theB-image may also be defined as a column direction, and the addingdirection of the correlation waveform may also be defined as a rowdirection.

When obtaining the optimal image shift amount I by calculating thecorrelation waveform Cm(I) in step S701 of FIG. 7 (when the calculationprocessing of the correlation waveform in FIG. 8 is ended) the flowproceeds to step S702. In step S702, the optimal image shift amount Icalculated in step S701 is converted to obtain a focus shift amount L.The image shift amount I can be converted into the focus shift amount Lby multiplying or adding coefficients for each F number. Subsequently,in step S703, a lens driving portion (not illustrated) of the imagepickup apparatus drives the lens (the image pickup lens) based on thefocus shift amount L calculated in step S702 and then the rangingcalculation processing is ended.

According to the present embodiment, an SN ratio is improved and thedefocus amount can be calculated by performing the weighting addition orthe interpolation of the saturated portion of correlation waveform whenadding the correlation waveform depending on each line in which thesaturated pixel is detected.

Embodiment 2

Next, an image pickup apparatus and a focus detection apparatus inEmbodiment 2 of the present invention will be described. In the presentembodiment, the descriptions with reference to FIGS. 1. to 10 are thesame as that of Embodiment 1, and therefore the descriptions thereofwill be omitted.

Referring to FIG. 12, an addition processing of a correlation waveform,in the present embodiment will be described. FIG. 12 is a flowchartillustrating the addition processing of the correlation waveform. Eachstep of FIG. 12 is performed based on an instruction of a controller ofthe image pickup apparatus (the focus detection apparatus).

First of all, in step S1201, the determination of the saturated line isperformed. In other words, in step S801 of FIG. 8, it is determinedwhether a saturated pixel is detected or not. In step S1201, when it isdetermined that the line is not the saturated line, i.e. when thesaturated pixel is not detected, the addition processing of thecorrelation waveform is performed in step S1202, as represented by thefollowing Expression (7).

Cm(I)=Cm(I)+Cm(Iy)  (7)

On the other hand, in step S1201, when it is determined that the line isthe saturated line, i.e. at least a predetermined number (apredetermined ratio) of the saturated pixels is detected, a value of SADof the correlation waveform for the saturated line is large, andtherefore the addition processing of the correlation waveform is notperformed. Thus, when at least the predetermined number (thepredetermined ratio) of the saturated pixels is included in a firstpixel array or the second pixel array of the first range, a controller1309 (an adding processor) perform de addition processing without usingthe correlation waveform in the first range. In other words, the addingprocessor performs the addition processing using the correlation dataobtained from the range in which the number of saturated pixels is lessthan a predetermined number.

As described above, in the present embodiment, when the correlationwaveform is added in accordance with the line in which the saturatedpixel is detected in a ranging region including the saturation pixel,the correlation waveform is normally added for the line where the number(a ratio) of the saturated pixels is less than a predetermined number (apredetermined ratio), and on the other hand, the correlation waveform isnot added for the line where the number (the ratio) of the saturatedpixels is not less than the predetermined number (the predeterminedratio). Therefore, a defocus amount in which the saturated line isexcluded to improve an SN ratio can be calculated.

(Application to Image Pickup System)

Next, referring to FIG. 13, an image pickup system to which the imagepickup apparatus in each embodiment is applicable will be described.FIG. 13 is a schematic diagram of the image pickup system 1300.

In FIG. 13, reference numeral 1301 denotes a lens portion (a lensapparatus) which forms an optical image of an object on an image pickupelement 1305 (a solid-state image pickup element). The lens portion 1301performs a zoom control, a focus control, a aperture stop control, orthe like, using a lens driving device 1302. Reference numeral 1303denotes a mechanical shutter. The mechanical shutter 1303 is controlledby a shutter controller 1304.

Reference numeral 1305 denotes an image pickup element which takes theobject imaged by the lens portion 1301, as an image signal. Referencenumeral 1306 denotes an image pickup signal processing circuit whichperforms various kinds of corrections for the image signal output fromthe image pickup element 1305 and compresses data. Reference numeral1307 denotes a timing generating circuit as a driving portion whichoutputs various kinds of timing signals to the image pickup element 1305and the image pickup signal processing circuit 1306. Reference numeral1309 denotes a controller which controls various kinds of calculationoperations and the entire image pickup apparatus. Reference numeral 1308is a memory portion (a storage portion) for temporarily storing imagedata. Reference numeral 1310 denotes an interface which records on orreads out from a recording medium. Reference numeral 1311 denotes adetachable recording medium, such as a semiconductor memory, which isconfigured to record or read out the image data. Reference numeral 1312denotes a display portion which displays various kinds of information orshot images.

Next, an image pickup operation of a digital camera an image pickupsystem 1300) in the configuration described above will be described.When a main power supply is turned on, a power supply of a controlsystem is turned on, and the power supply of an image pickup systemcircuit such as the image pickup signal processing circuit 1306 isturned on. Subsequently, when a release button (not illustrated) ispressed, the controller 1309 performs a ranging calculation based on thedata from the image pickup element 1305 and calculates a distance to theobject based on the ranging result. Then, the lens driving device 1302drives the lens portion 1301 and determines whether the focus state isthe in-focus state. When it is determined that the focus state is notthe in-focus state, the lens portion 1301 is again driven to perform theranging (the focus detection). The ranging calculation is obtained bythe data from the image pickup element 1305, or alternatively may alsobe performed by a dedicated ranging apparatus (not illustrated).

Then, an image shooting operation starts after the in-focus state isconfirmed. When the image shooting operation ends, the image pickupsignal processing circuit 1306 performs an image processing for theimage signal output from the image pickup element 1305, and thecontroller 1309 writes the processed image signal in the memory portion1308. The image pickup signal processing circuit 1306 performsrearrangement processing, addition processing, and selection processingthereof. The data accumulated in the memory portion 1308 is recorded onthe detachable recording medium 1311, such as a semiconductor memory,through a recording medium control I/F portion 1310 according to thecontrol by the controller 1309. In addition, the data may be directlyinput to the computer or the like through an external I/F portion (notillustrated) to perform the image processing. The image pickup system1300 is configured to include the image pickup apparatus (the imagepickup apparatus body) and the lens portion 1301 (a lens apparatus)removably mounted on the image pickup apparatus, and the image pickupapparatus controls the driving of the lens portion 1301 based on thesignal obtained from the focus detection apparatus in each embodiment.Each embodiment is not limited to this, and the image pickup apparatusbody and the lens portion may be integrally configured.

According to each embodiment, a focus detection apparatus, an imagepickup apparatus, an image pickup system, and a focus detection methodcapable of reducing the deterioration of the focus detection accuracyeven when a saturated pixel is included can be provided.

Particularly, the image pickup apparatus of each embodiment uses eachsignal (independent signals) of the first. pixel (A-image pixel) and thesecond pixel (B-image pixel) of image pickup apparatus as an imagedetection signal, and uses the adding signal of the first pixel and thesecond pixel as an image pickup signal. In order to obtain a shot imagewith higher quality in such apparatus, the shot image with high qualitycan be obtained by actively moving electric charges to the other sidewhen one of the first pixel or the second pixel approaches thesaturation. Therefore, it is preferred that the image pickup elementadopts a structure which is configured, for example, to lower thepotential between the first pixel and the second pixel. Such structureincreases a possibility that both the first pixel and the second pixelare saturated. When photoelectric conversion portions of both the firstpixel and the second pixel are saturated, it may be a greatly erroneousfactor when the focus detection is performed. Therefore, when thesaturated pixel is included in both the first pixel array and the secondpixel array, the adding processor preferably performs the additionprocessing so as to reduce influence of the correlation waveformcalculated in the first range. In this way, each embodiment isparticularly effective to the image pickup apparatus (the image pickupelement) having the configuration described above.

Furthermore, the following embodiment may be also applied. Based on thedivided PDs 203 and 204 sharing the same microlens 202, it is possibleto add the accumulated electric charges to read out, and the accumulatedelectric charges can be also read out selectively on each divided PDs203 and 204 in non-destructive manner. At this time, first of all, fromthe image pickup element 100, a non-destructive readout of the A-imagepixel signals for one horizontal line, that is, a nondestructive readoutof the divided pixel signals from the divided PD 203 is performed.Subsequently, from the image pickup element 100, the readout of theadding signal of the A-image pixel signal and the B-image pixel signalon the same line, that is, the readout of a unit pixel signal from thedivided PDs 203 and 204 is performed. Then, the controller 1309illustrated in FIG. 13 performs the saturation detection of apredetermined value or more as in step S801 and the readout control ofthese signals from the image pickup element 100. At this time, theA-image signal serving as the pixel signal is output to the image pickupsignal processing circuit 1306 at the time of the readout of the A-imagepixel, and the image pickup signal processing circuit 1306 detects thesaturation of the A-image pixel. On the other hand, (A+B) image signalserving as the pixel signal is output to the image pickup signalprocessing circuit 1306 at the time of the readout of the capturingpixel. The brightness signal of the B-image pixel can be obtained by adifference between the brightness signal obtained from the pixel signalof the A-image pixel corresponding to a plurality of color filters andthe (A+B) image, and the saturation detection of the B-image pixelsignal is detected by the saturation detection of the brightness signalof the B-image pixel. By the saturation detection described above, thesaturation detection of the A-image pixel signal, the saturationdetection of the B-image pixel signal, and the saturation detection ofthe (A+B) image signal can be performed, and thus the error of thecorrelation waveform due to any cause can be also suppressed.

Other Embodiments

The purpose of the present invention can be achieved even when thefollowing embodiment is applied. That is, the invention provides asystem or apparatus with a non-transitory computer-readable storagemedium that stores program codes of software described with a procedurefor realizing the function of each embodiment described above. In thisway, the computer (CPU, MPU, or the like) of the system or apparatusexecutes operation by reading out the program codes that are stored inthe storage medium.

In this case, the program codes itself read out from the storage mediumimplement the novel functions of the invention, and thus the program andthe storage medium that stores the program codes are included in theconfiguration of the present invention.

As the storage medium for providing the program codes, for example,there are a flexible disk, a hard disk, an optical disk, amagneto-optical disk, and the like. In addition, a CD-ROM, a CD-R, aCD-RW, a DVD-ROM, a DVD-RAM, a DVD-RW, a DVD-R, a magnetic tape, anon-volatile memory card, a ROM and the like can be used as the storagemedium.

When the program codes read out by the computer are executable, thefunctions of each embodiment described above are implemented.Furthermore, a case in which the OS (operating system) running on thecomputer and the like performs a part or all of actual processing basedon the instructions of the program codes to implement the functions ofeach embodiment described above is also included.

In addition, a following case is also included. First, the program,codes read out from the storage medium are written in a memory providedin a function expansion unit. connected to the computer or a functionexpansion board inserted in the composer. Thereafter, based on theinstructions of the program codes, the CPU and the like provided in thefunction expansion board and the function expansion unit performs a partor all of actual processing.

Furthermore, the present invention is applicable not only to theapparatus such as a digital camera which is used for the capturing asmain purpose, but also to an arbitrary apparatus which is embedded orconnected with respect to the image pickup apparatus such as a cellularphone, a personal computer (laptop-type, desktop-type, tablet-type, orthe like), and a game machine. Accordingly, in this specification, the“image pickup apparatus” is intended to include an arbitrary electronicapparatus provided with the image pickup function.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2012-261392, filed on Nov. 29, 2012, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. A focus detection apparatus which performs focusdetection by a phase difference method using an image pickup elementincluding a plurality of first pixels and a plurality of second pixels,each pair of the first and second pixels sharing one microlens, thefocus detection apparatus comprising: a correlation data calculatorconfigured to calculate correlation data between pixel data obtainedfrom the plurality of first pixels and pixel data obtained from theplurality of second pixels in a plurality of ranges of image dataobtained from the image pickup element; a detector configured to detecta saturated pixel having a level of at least a predetermined value ineach of the plurality of ranges; an adding processor configured toperform an addition processing of the correlation data calculated ineach of the plurality of ranges, based on a detection result of thesaturated pixel by the detector; and a defocus amount calculatorconfigured to calculate a defocus amount based on a result of theaddition processing, wherein the adding processor performs the additionprocessing using correlation data obtained from a first range in whichthe number of the saturated pixels is less than a predetermined number.2. The focus detection apparatus according to claim 1, wherein theadding processor uses the correlation data of the first pixels or thesecond pixels in a second range adjacent to the first range as thecorrelation data in the first range, for the first range in which thenumber of the saturated pixels among the first pixels or the secondpixels is not less than the predetermined number.
 3. The focus detectionapparatus according to claim 2, wherein when the saturated pixel isincluded in the first pixels or the second pixels, the adding processoruses the correlation data in the second range adjacent to the firstrange as the correlation data in the first range, for a range in whichthe saturated pixel exists in the first range.
 4. The focus detectionapparatus according to claim 1, wherein when the saturated pixel isincluded in a first pixel array including the plurality of first pixelsor a second pixel array including the plurality of second pixels, theadding processor performs the addition processing by multiplying aweight depending on she number of the saturated pixels by thecorrelation data in the first range.
 5. An image pickup apparatuscomprising the focus detection apparatus according to claim
 1. 6. Theimage pickup apparatus according to claim 5, wherein the first pixel andthe second pixel receive light beams passing through different regionsof an exit pupil different from each other in a pupil dividing directionso as to generate signals to be used to calculate the correlation data,and wherein the adding processor adds the correlation data of each ofthe plurality of ranges in a direction orthogonal to the pupil dividingdirection.
 7. An image pickup system comprising: an image pickupapparatus according to claim 5; and a lens apparatus configured to beremovably mounted on the image pickup apparatus, wherein the imagepickup apparatus performs a drive control of the lens apparatus based ona signal obtained from the focus detection apparatus.
 8. A focusdetection method of performing focus detection by a phase differencemethod using an image pickup element including a plurality of firstpixels and a plurality of second pixels, each pair of the first andsecond pixels sharing one microlens, the focus detection methodcomprising the steps of: calculating correlation data between pixel dataobtained from the plurality of first pixels and pixel data obtained fromthe plurality of second pixels in a plurality of ranges of image dataobtained from the image pickup element; detecting a saturated pixelhaving a level of at least a predetermined value in each of theplurality of ranges; performing an addition on processing of thecorrelation data calculated in each of the plurality of ranges, based ona detection result of the saturated pixel; and calculating a defocusamount based on a result of the addition processing, wherein theaddition processing is performed using correlation data obtained from arange in which the number of the saturated pixels is less than apredetermined number.
 9. A non-transitory computer-readable storagemedium which stores a program to cause a computer to execute each stepof the focus detection method according to claim 8.